<html>
<head><meta charset="utf-8"><title>Handling of &#x27;devirtualized&#x27; paths · t-compiler/wg-incr-comp · Zulip Chat Archive</title></head>
<h2>Stream: <a href="https://rust-lang.github.io/zulip_archive/stream/241847-t-compiler/wg-incr-comp/index.html">t-compiler/wg-incr-comp</a></h2>
<h3>Topic: <a href="https://rust-lang.github.io/zulip_archive/stream/241847-t-compiler/wg-incr-comp/topic/Handling.20of.20&#x27;devirtualized&#x27;.20paths.html">Handling of &#x27;devirtualized&#x27; paths</a></h3>

<hr>

<base href="https://rust-lang.zulipchat.com">

<head><link href="https://rust-lang.github.io/zulip_archive/style.css" rel="stylesheet"></head>

<a name="233331863"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/241847-t-compiler/wg-incr-comp/topic/Handling%20of%20%27devirtualized%27%20paths/near/233331863" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Aaron Hill <a href="https://rust-lang.github.io/zulip_archive/stream/241847-t-compiler/wg-incr-comp/topic/Handling.20of.20&#x27;devirtualized&#x27;.20paths.html#233331863">(Apr 06 2021 at 14:53)</a>:</h4>
<p>When we load <code>SourceFile</code>s from an external crate, we try to 'devirtualized' paths that point into the standard library: <a href="https://github.com/rust-lang/rust/blob/e1d49aaad4fde33469fdb786c29838a95a5d8a11/compiler/rustc_metadata/src/rmeta/decoder.rs#L1633-L1682">https://github.com/rust-lang/rust/blob/e1d49aaad4fde33469fdb786c29838a95a5d8a11/compiler/rustc_metadata/src/rmeta/decoder.rs#L1633-L1682</a><br>
The result of the devirtualiztion depends on global untracked state (whether or not the <code>rust-src</code> exists during the current compilation session), represented by <code>sess.real_rust_source_base_dir</code>. As a result, adding/removing the <code>rust-src</code> compoonent can lead to an ICE now that we verify query result hashes.</p>
<p>Currently, all external queries depend on the <code>eval_always</code> query <code>crate_hash</code>, which ensures that they get invalidated if an upstream crate is changed. I think the simplest way to fix this issue would be to make <code>crate_hash</code> depend on <code>real_rust_source_base_dir</code>. Unfortunately, this would mean that the crate computed seen during compilation of a crate is not necessarily the same as the crate hash that gets seen by a downstream crate.</p>
<p>Another alternative would be to make a new query <code>crate_dependecy(CrateNum) -&gt; (Svh, String)</code> that explcitily includes <code>real_rust_source_base_dir</code> in its result. This would then cause external queries to be re-computed as necessary, without the need to modify the crate hash.</p>
<p>I don't really like either of these solutions - I'm open to any other suggestions. The root issue is that data loaded from <em>foreign</em> crates depends on the state of our current compilation session,, but there's nothing we can do about that.</p>



<a name="233335061"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/241847-t-compiler/wg-incr-comp/topic/Handling%20of%20%27devirtualized%27%20paths/near/233335061" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> bjorn3 <a href="https://rust-lang.github.io/zulip_archive/stream/241847-t-compiler/wg-incr-comp/topic/Handling.20of.20&#x27;devirtualized&#x27;.20paths.html#233335061">(Apr 06 2021 at 15:04)</a>:</h4>
<p>I think we should keep using the virtualized path everywhere and only use the devirtualized path to actually read the source files. Currently builds are not reproducible with <code>rust-src</code> installed or not installed as it leaks into the debuginfo as far as I know.</p>



<a name="233335332"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/241847-t-compiler/wg-incr-comp/topic/Handling%20of%20%27devirtualized%27%20paths/near/233335332" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Aaron Hill <a href="https://rust-lang.github.io/zulip_archive/stream/241847-t-compiler/wg-incr-comp/topic/Handling.20of.20&#x27;devirtualized&#x27;.20paths.html#233335332">(Apr 06 2021 at 15:05)</a>:</h4>
<p>I thought the leaking in the debuginfo was intentional, so that you get nice-looking paths in panic messages</p>



<a name="233335463"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/241847-t-compiler/wg-incr-comp/topic/Handling%20of%20%27devirtualized%27%20paths/near/233335463" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Aaron Hill <a href="https://rust-lang.github.io/zulip_archive/stream/241847-t-compiler/wg-incr-comp/topic/Handling.20of.20&#x27;devirtualized&#x27;.20paths.html#233335463">(Apr 06 2021 at 15:06)</a>:</h4>
<p>is there another use-case for devirtualized paths?</p>



<a name="233335630"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/241847-t-compiler/wg-incr-comp/topic/Handling%20of%20%27devirtualized%27%20paths/near/233335630" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> bjorn3 <a href="https://rust-lang.github.io/zulip_archive/stream/241847-t-compiler/wg-incr-comp/topic/Handling.20of.20&#x27;devirtualized&#x27;.20paths.html#233335630">(Apr 06 2021 at 15:06)</a>:</h4>
<p>Devirtualized paths are used to load the source for sysroot crates to show in error messages.</p>



<a name="233335776"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/241847-t-compiler/wg-incr-comp/topic/Handling%20of%20%27devirtualized%27%20paths/near/233335776" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> bjorn3 <a href="https://rust-lang.github.io/zulip_archive/stream/241847-t-compiler/wg-incr-comp/topic/Handling.20of.20&#x27;devirtualized&#x27;.20paths.html#233335776">(Apr 06 2021 at 15:07)</a>:</h4>
<p>Maybe the debuginfo was also intentional, but I don't like it.</p>



<a name="233335941"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/241847-t-compiler/wg-incr-comp/topic/Handling%20of%20%27devirtualized%27%20paths/near/233335941" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Aaron Hill <a href="https://rust-lang.github.io/zulip_archive/stream/241847-t-compiler/wg-incr-comp/topic/Handling.20of.20&#x27;devirtualized&#x27;.20paths.html#233335941">(Apr 06 2021 at 15:07)</a>:</h4>
<p>unfortunately, I think we cache diagnostics as part of a result</p>



<a name="233336008"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/241847-t-compiler/wg-incr-comp/topic/Handling%20of%20%27devirtualized%27%20paths/near/233336008" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Aaron Hill <a href="https://rust-lang.github.io/zulip_archive/stream/241847-t-compiler/wg-incr-comp/topic/Handling.20of.20&#x27;devirtualized&#x27;.20paths.html#233336008">(Apr 06 2021 at 15:08)</a>:</h4>
<p>and you could potentially emit a warning with a span that points into a devirtualized path</p>



<a name="233336097"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/241847-t-compiler/wg-incr-comp/topic/Handling%20of%20%27devirtualized%27%20paths/near/233336097" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Aaron Hill <a href="https://rust-lang.github.io/zulip_archive/stream/241847-t-compiler/wg-incr-comp/topic/Handling.20of.20&#x27;devirtualized&#x27;.20paths.html#233336097">(Apr 06 2021 at 15:08)</a>:</h4>
<p>so just using devirtualized paths for reading source files could still leak untracked global state into the query system</p>



<a name="233336685"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/241847-t-compiler/wg-incr-comp/topic/Handling%20of%20%27devirtualized%27%20paths/near/233336685" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Aaron Hill <a href="https://rust-lang.github.io/zulip_archive/stream/241847-t-compiler/wg-incr-comp/topic/Handling.20of.20&#x27;devirtualized&#x27;.20paths.html#233336685">(Apr 06 2021 at 15:10)</a>:</h4>
<p>not to mention the fact that that Cargo is unaware of the existence of <code>rustc-src</code>, so you could have a mixture of devirtualized and virtualized paths in the same crate graph</p>



<a name="233336953"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/241847-t-compiler/wg-incr-comp/topic/Handling%20of%20%27devirtualized%27%20paths/near/233336953" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Aaron Hill <a href="https://rust-lang.github.io/zulip_archive/stream/241847-t-compiler/wg-incr-comp/topic/Handling.20of.20&#x27;devirtualized&#x27;.20paths.html#233336953">(Apr 06 2021 at 15:11)</a>:</h4>
<p>I wonder if we could deprecate and remove the automatic detection, and require explictly passing in the path via a command-line argument</p>



<a name="233337020"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/241847-t-compiler/wg-incr-comp/topic/Handling%20of%20%27devirtualized%27%20paths/near/233337020" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> bjorn3 <a href="https://rust-lang.github.io/zulip_archive/stream/241847-t-compiler/wg-incr-comp/topic/Handling.20of.20&#x27;devirtualized&#x27;.20paths.html#233337020">(Apr 06 2021 at 15:11)</a>:</h4>
<p><span class="user-mention silent" data-user-id="125294">Aaron Hill</span> <a href="#narrow/stream/241847-t-compiler.2Fwg-incr-comp/topic/Handling.20of.20'devirtualized'.20paths/near/233336097">said</a>:</p>
<blockquote>
<p>so just using devirtualized paths for reading source files could still leak untracked global state into the query system</p>
</blockquote>
<p>I mean delaying the devirtualization until the <code>fs::read_to_string</code>, so the <code>SourceMap</code> would still contain the virtualized paths.</p>



<a name="233337103"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/241847-t-compiler/wg-incr-comp/topic/Handling%20of%20%27devirtualized%27%20paths/near/233337103" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Aaron Hill <a href="https://rust-lang.github.io/zulip_archive/stream/241847-t-compiler/wg-incr-comp/topic/Handling.20of.20&#x27;devirtualized&#x27;.20paths.html#233337103">(Apr 06 2021 at 15:12)</a>:</h4>
<p>hmm</p>



<a name="233337364"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/241847-t-compiler/wg-incr-comp/topic/Handling%20of%20%27devirtualized%27%20paths/near/233337364" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Aaron Hill <a href="https://rust-lang.github.io/zulip_archive/stream/241847-t-compiler/wg-incr-comp/topic/Handling.20of.20&#x27;devirtualized&#x27;.20paths.html#233337364">(Apr 06 2021 at 15:13)</a>:</h4>
<p>that could work - we'd want some kind of wrapper around the <code>RealFileName</code> fields to ensure that a query can't use the actual path bytes</p>



<hr><p>Last updated: Aug 07 2021 at 22:04 UTC</p>
</html>